On Nash Equilibria in Stochastic Games

We study in nite stochastic games played by n-players on a nite graph with goals given by sets of in nite traces. The games are stochastic (each player simultaneously and independently chooses an action at each round, and the next state is determined by a probability distribution depending on the current state and the chosen actions), innite (the game continues for an in nite number of rounds), nonzero sum (the players' goals are not necessarily con icting), and undiscounted. We show that if each player has a reachability objective, that is, if the goal for each player i is to visit some subset Ri of the states, then there exists an -Nash equilibrium in memoryless strategies. We study the complexity of nding such Nash equilibria. Given an n-player reachability game, and a vector of values (v1; : : : ; vn), we show it is NP-hard to determine if there exists a memoryless -Nash equilibrium where each player gets payo at least vi. On the other hand, for every xed , the value can be -approximated in FNP. We study two important special cases of the general problem. First, we study n-player turn-based probabilistic games, where at each state atmost one player has a nontrivial choice of moves. For turn-based probabilistic games, we show the existence of -Nash equilibria in pure strategies for all games where the goal of each player is a Borel set of in nite traces. We also derive the existence of pure exact Nash equilibria for n-player turn-based games where each player has an !-regular objective. Then we study the two player case and show that already for two-player games exact Nash equilibria may not exist. Our techniques for the general case also yield NP \ coNP -approximation algorithms for zero-sum reachability games, improving the previously known EXPTIME bound.